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(54) Apparatus for receiving broadcast data, method for displaying broadcast program, and 
computer program 



(57) A data broadcasting receiver apparatus in- 
cludes a receiver for receiving image data of the image 
of an object and size information relating to the object, 
transmitted through data broadcasting, an input unit for 
inputting screen information relating to a display screen 
of a display, a scaling unit for determining a scale factor 



of the object image based on the screen information in- 
put by the input unit and the size information received 
by the receiver so that the object image is displayed on 
the display at a predetermined size, and a display con- 
trol unit for controlling the display to display the object 
image expanded or contracted at the scale factor deter- 
mined by the scaling unit. 



FIG. 1 



CM 
< 
CNJ 

00 

CO 
CO 



TUNER 




(^SCRAMBLER 




TRANSPORT 




AUDIO 
DECODER 






DECODER 







\ 
lOt 



113 



OPERATION 
UNIT 



102 



112 



MEMORY 



103 



110 



106 



_ VIDEO 
DECODER 



GRAPHIC 
GENERATOR 



-104 



121 



107 



MOVING IMAGE PLANE 



STILL IMAGE PLANE 



CHARACTER AND 
.DRAWING IMAGE PLANE 



109 



108 



1^ 



VCR 



3 



115 



PHOTOSENSOR 



116 



ICCARD 
CONTROLLER 



117 



FT 



CPU 



118 



HARD 
DISK 



119 



DAC 


1^105 




LOUDSPEAKER 







111 



IMAGE 
SYNTHESIZER 



SERIAL 
l/F 



120 



^130 



PRINTER 
V 131 



REMOTE 
CONTROLLER 



^7 



32 



133 



DISPLAY 



134 



✓-100 



Q. 
LU 



Printed by Jouve, 75001 PARIS (FR) 



£1 



1 



EP 1 331 812 A2 



Descrfption 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001 ] The present invention relates to a display con- 
trol technique of data broadcast content in which char- 
acters, image, voice, video are multiplexed. 

Description of the Related Art 

[0002] As digital satellite broadcasting service has 
started, data-broadcasting service is received by not on- 
ly personal computers but also television receivers. With 
the television receiver having a receiving function of re- 
ceiving data broadcast data and a replay software pro- 
gram (a data broadcast browser), even a user who has 
no personal computer views a variety of pieces of infor- 
mation and displays texts and images, associated with 
a television program, on the screen of the television re- 
ceiver. Standards controlling such a service is Standard 
ARIB-STD-B24 formulated by the Association of Radio 
Industries and Business (ARIB) in Japan. 
[0003] TV stations encodes ordinary TV program da- 
ta, audio data, and data broadcasting data. After being 
multiplexed, these pieces of data are modulated and 
then transmitted from an antenna. A signal transmitted 
from the antenna is then delivered to a user terminal (a 
television receiver) through a broadcasting satellite 
(BS). 

[0004] FIG. 1 7 is a block diagram showing the con- 
struction of a typical receiver for receiving typical digital 
satellite broadcast signals. The Standard of the ARIB 
permits the resolution of a video plane 107, still image 
plane 1 08, and character and drawing image plane 1 09 
to be different from receiver to receiver. For example, 
the still image plane 1 08 may have two types of resolu- 
tions, namely, a resolution of 1920 pixels (horizontally) 
by 1 080 lines (vertically), and a resolution of 720 pixels 
(horizontally) by 480 lines (vertically). With the still im- 
age plane 108 of a resolution of 720 x 480, an aspect 
ratio of width frame to height frame may be 1 6:9 or 4:3 
on a display. 

[0005] With the still image plane 108 of a resolution 
of 1920 x 1080, each pixel on the still image plane 1 08 
is displayed as a square. But with a resolution of 720 x 
480, each pixel of the still image plane 1 08 has a slightly 
horizontally elongated rectangular shape if the aspect 
ratio of the display is 16:9, and has a slightly vertically 
elongated rectangular shape if the aspect ratio of the 
display is 4:3. The display is available in a variety of siz- 
es, for example, 14 inches, 19 inches, and 33 inches. 
[0006] Digital data is repeatedly transmitted from TV 
stations in the data carousel method standardized in 
ISO/IEC1381 8-6. Digital data, filtered by a transport de- 
coder, contains text information, script information, im- 
age information, and video and audio data. Text infor- 



mation is described using XML (extensible Markup Lan-. 
guage) regulated by the Worldwide Web Consortium 
(W3C). 

[0007] Attempts to provide services such as electron- 
5 ic shopping using data broadcasting have been made. 
For example, when a television-shopping program is in 
progress, detailed information about goods is delivered 
using data broadcasting. A user operates the television 
receiver to monitor the detailed information about a 
10 commercial product currently being introduced during 
the program. 

[0008] In the electronic shopping using the television 
receiver, the size of the displayed image of each com- 
modity becomes different depending on the size of a TV 
15 screen in use and the resolution of the TV. The real size 
of each commodity and the size of the commodity which 
users feel according to the displayed image fail to match 
each other. 

[0009] Japanese Laid-Open Patent No. 9-006944 dis- 

20 closes an electronic catalog system to reproduce the 
size of the commodity. According to the disclosure, the 
electronic catalog system scales the image of a product 
in accordance with the size of a display screen to 
present the image of the product in the real size thereof. 

25 [0010] As described above, however, the receiving 
environment such as the resolution of the still image 
plane 108 of the TV receiver, the size of the display, and 
the aspect ratio of the display screen during presenta- 
tion may be different from user to user. 

so [0011] The technique, disclosed in the above-cited 
Japanese Laid-Open Patent No. 9L-006944, simply per- 
forms a scaling process taking into consideration the 
screen size only. The aspect ratio of an original product 
may change depending on the specifications of a TV re- 

35 ceiver of a user. This technique is unable to present a 
product in its real size, and thus it is difficult to cause the 
impression of the product on the TV display screen to 
the user to match the actual impression of the product. 
[0012] In accordance with the cited disclosure, the us- 

40 er needs to input data of the screen size, and the image 
data of the product in the real size thereof is generated 
from particular image data. Depending on a combination 
of the size of a product, display screen size, and reso- 
lution, the image quality of the image of the product is 

45 substantially degraded during real-size presentation. 
On-screen realism of the product is thus damaged. 
[0013] For example, if a product as large as 1 meter 
high is displayed on a 1 4-inch TV monitor in the real size 
thereof as shown in FIG. 18A, a portion of the product 

so is conventionally displayed in enlargement. The image 
quality is thus degraded. 

[0014] In the example shown in FIGS. 18A and 18B, 
the image of a "bicycle" as long as 1 .5 m is presented 
in the real size thereof on a 14-inch size TV screen. A 
55 portion of the image shown in FIG. 1 8A is thus presented 
as shown in FIG. 18B. If the resolution of the image 
shown in FIG. 18A is not high enough, image quality 
substantially decreases as shown in FIG. 18B. Realism 
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of the bicycle is lost. 

SUMMARY OF THE INVENTION 

[0015] It is a concern of the present invention to pro- 
vide an apparatus, method and computer program for 
displaying an image of an object in a shape reflecting 
the size of the original object from image data transmit- 
ted through data broadcasting, regardless of the speci- 
fications of a receiver. 

[0016] In an aspect, the present invention relates to 
an apparatus for receiving broadcast data, and includes 
a receiver for receiving image data of the image of an 
object and size information relating to the object, trans- 
mitted through data broadcasting, an input unit for in- 
putting screen information relating to a display screen 
of display, a scaling unit for determining a scale factor 
of the object image based on the screen information in- 
put by the input unit and the size information received 
by the receiver so that the object image is displayed on 
the display at a predetermined size, and a display con- 
trol unit for controlling the display to display the object 
image expanded or contracted at the scale factor deter- 
mined by the scaling unit. 

[0017] Further features, and advantages of the 
present invention will be apparent from the following de- 
scription of the preferred embodiments with reference 
to the attached drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0018] FIG. 1 is a block diagram showing the con- 
struction of a TV receiver incorporating the present in- 
vention. 

[0019] FIG. 2 shows a software configuration running 
on a CPU 118. 

[0020] FIG. 3 is a flow diagram showing the content 
of a process of a data broadcasting browser. 
[0021 ] FIG. 4 shows an example of XML data for star- 
tup screen in accordance with a first embodiment of the 
present invention. 

[0022] FIG. 5 is a flow diagram of an initial script proc- 
ess described In the XML data. 
[0023] FIG. 6 shows a display screen which is gener- 
ated by the data broadcasting browser from the XML 
data of the startup screen. 

[0024] FIGS. 7A and 7B show display screens select- 
ed when a general display button is selected on the 
screen shown in FIG. 6. 

[0025] FIGS. 8A and 8B show display screens select- 
ed when a texture display button is selected on the 
screen shown in FIG. 6. 

[0026] FIG. 9 shows an example of XML data of a real- 
size display screen. 

[0027] FIG. 10 is a continuation of the XML data of 
FIG. 9. 

[0028] FIG. 11 is a flow diagram of an initial script 
process described in the XML data shown in FIGS. 9 



and 10. 

[0029] FIG. 1 2 is a flow diagram showing, in detail, a 
calculation process of a real-size display parameter of 
FIG. 11. 

5 [0030] FIG. 13 is a flow diagram showing, in detail, an 
appropriateness determination process of FIG. 11 . 
[0031] FIG. 1 4 is a flow diagram showing, in detail, a 
moving image window adjustment process of FIG. 11 . 
[0032] FIGS. 15A and 15B show an example of dis- 
10 play screen which is generated by the data broadcasting 
browser from the XML data of the real-size screen. 
[0033] FIGS. 1 6A and 1 6B show examples of display 
screen which is generated by the data broadcasting 
browser from the XML data of the real-size screen. 
is [0034] FIG. 17 is a block diagram showing the con- 
struction of a typical digital TV receiver. 
[0035] FIG. 18 shows a screen on which a real-size 
presentation is performed using a conventional art. 
[0036] FIG. 1 9 shows XML data of a startup screen in 
accordance with a second embodiment of the present 
invention. 

[0037] FIG. 20 is a continuation of the XML data of 
FIG. 19. 

[0038] FIG. 21 is a flow diagram showing an initial 
script process described in the XML data. 
[0039] FIG. 22 shows a display screen which is gen- 
erated by a data broadcasting browser from the XML 
data of a startup screen. 

[0040] FIGS. 23A and 23B show a display screen with 
a general display button selected on the screen of FIG. 
22. 



[0041] 


FIG. 


24 


shows XML data of a 


virtual fitting 


screen. 










[0042] 


FIG. 


25 


is a continuation of the 


XML data of 


FIG. 24. 










[0043] 


FIG. 


26 


is a continuation of the 


XML data of 


FIG. 25. 










[0044] 


FIG. 


27 


is a continuation of the 


XML data of 


FIG. 26. 










[0045] 


FIG. 


28 


is a continuation of the 


XML data of 


FIG. 27. 










[0046] 


FIG. 


29 


is a continuation of the 


XML data of 


FIG. 28. 










[0047] 


FIG. 


30 


is a continuation of the 


XML data of 



FIG. 29. 

[0048] FIG. 31 is a flow diagram showing a script proc- 
ess described in the XML data of the virtual fitting screen 
shown in FIGS. 24 through 30. 
[0049] FIG . 32 is a flow diagram showing, in detail, an 
adjustment process of a virtual fitting image (synthe- 
sized image) in FIG. 31 . 

[0050] FIGS. 33A and 33B respectively show an im- 
age of a product transmitted through data broadcasting 
and an image of a user registered in a digital TV receiver. 
[0051] FIGS. 34A-34C show virtual fitting images 
(synthesized images). 

[0052] FIGS. 35A-35C show other virtual fitting imag- 
es (synthesized images). 
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DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0053] The embodiments of the present invention will 
now be discussed with reference to the drawings. 5 
[0054] FIG. 1 is a block diagram showing the con- 
struction of a digital TV receiver apparatus 1 00 incorpo- 
rating the present invention. 

[0055] Referring to FIG. 1 , a signal received by an an- 
tenna is input to a tuner 101 . The tuner 101 performs io 
processes, such as demodulation and error correction, 
on the input signal, and generates digital data referred 
to as transport stream (TS). The transport stream (TS) 
data is output to a descrambler 102. 
[0056] Upon receiving, from the tuner 101, the TS da- '5 
ta that is scrambled to restrict viewing, the descrambler 
1 02 descrambles the TS data based on key information 
contained in the TS data for descrambling and key in- 
formation output from an IC card controller 1 1 7, and out- 
puts descrambled TS data to a transport decoder 1 03. 20 
[0057] The IC card controller 1 1 7 contains an IC card 
which stores user contract information and key informa- 
tion for decrypting the key information for descrambling 
contained in theTS data. When the key information for 
decrypting the key information for descrambling is 25 
present, the IC card controller 1 1 7 outputs the key infor- 
mation to the descrambler 1 02. 
[0058] Upon receiving, from the tuner 101, the un- 
scrambled TS data, the descrambler 1 02 outputs the TS 
data to the transport decoder 1 03. The transport decod- 30 
er 1 03 extracts a packet corresponding to a program se- 
lected by the user. 

[0059] The video data in the extracted packet is de- 
coded by a video decoder 106, and is then sent to a 
moving image plane 107 in a graphic buffer 121. The 35 
audio data in the extracted packet is decoded by an au- 
dio decoder 104, and is then output to a loudspeaker 
133 through a D/A converter 105. Data broadcast data 
in the extracted packet is fed to a memory 112 through 
a bus 1 22 connected to a CPU 1 18. The data broadcast 40 
data, after being decoded by the CPU 118, is stored in 
a hard disk 119. 

[0060] When the data broadcast data is displayed, the 
CPU 118 reads the data broadcast data stored in the 
hard disk 119, converts the data broadcast data into im- « 
age data of characters and drawings using a graphic 
generator 1 1 0, arid then sends the image data to one of 
character and drawing image plane 109 and still image 
plane 1 08 in the graphic buffer 1 21 . 
[0061] The character and drawing image plane 109 so 
chiefly stores bit images such as buttons, texts, draw- 
ings, and pictures, serving as Ul (user interface) in data 
broadcasting. The still image plane 108 stores bit imag- 
es of photographs. 

[0062] An image synthesizer 111 synthesizes and 55 
switches graphic data stored in each of the moving im- 
age plane 1 07, character and drawing image plane 1 09, 
and still image plane 108 in the graphic buffer 121, and 
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outputs the resulting data to a display 134. 
[0063] The display 134 displays the incoming image 
data. The display 1 34 has a memory storing information, 
such as a screen size of the display 134. Information 
stored in the memory is sent to the digital TV receiver 
apparatus 100 through a serial bus connected between 
the digital TV receiver apparatus 100 and display 134, 
and is thus sent to and stored in the memory 112 con- 
trolled by the CPU 118 through a serial interface 120 
and bus 122. 

[0064] Also connected to the bus 122 are an 
IEEE1394 interface 114 and a modem 115. The 
IEEE1394 interface 114 is used for the digital TV receiv- 
er apparatus 1 00 to perform protocol communication 
with an external VCR (Video Cassette Recorder) 130 
and a printer 131. The modem 115 is used to be con- 
nected to the Internet through a telephone line. 

Data Broadcasting 

[0065] In the data broadcasting, digital data is repeat- 
edly transmitted from TV stations in the DSM-CC data 
carousel method formulated in standard ISO/ 
IEC13818-6. Digital data, filtered by the transport de- 
coder 103, contains text information, script information, 
image information, and video and audio data. Text infor- 
mation is described using XML (extensible Markup Lan- 
guage) regulated by the W3C. 
[0066] This embodiment employs specification based 
on XHTML specification which is standardized by refor- 
mulating HTML4.0 with XML1 .0. In XML, a character se- 
quence in a document is attributed to (defined by) a de- 
scription between tag <> and tag </>. The tag may have 
a nested structure. The format is CSS (Cascading Style 
Sheet), and the script process uses Java (Trade Name) 
Script. 

[0067] In this embodiment, the data broadcasting is 
linked with a shopping program. An object image cap- 
tured from a product, being currently introduced in the 
shopping program, is multiplexed on the data broadcast 
data as a still image, and is then transmitted from a TV 
station. A plurality of types of object image data of a sin- 
gle product, such as a general image of the product, and 
an image that is obtained by enlarging a portion of the 
product to express texture of the product, are multi- 
plexed to the data broadcast data. 
[0068] FIG. 2 shows a software configuration operat- 
ing on the CPU 118. A data broadcasting browser 201 
is an application running on an operating system 200. 
The data broadcasting browser 201 controls the graphic 
generator 1 1 0 using a graphics library 202 and graphics 
driver 203, thereby rendering characters, drawings, and 
image on the graphic buffer 121 . 
[0069] The data broadcasting browser 201 also con- 
trols the image synthesizer 111 using a window control 
middleware 204 and image synthesis driver 205, there- 
by synthesizing moving images, still images, charac- 
ters, and drawings. The data broadcasting browser 201 
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controls the transport decoder 1 03 using a transport de- 
coder driver 206, thereby performing filter setting on a 
moving image and voice to be reproduced, and receiv- 
ing data broadcast data. The data broadcasting browser 
201 further controls the serial interface 120 using a se- 5 
rial driver 207, thereby reading the screen size and as- 
pect ratio of the display 134. The data broadcasting 
browser 201 communicates with a remote controller 1 32 
through a remote controller driver 208, thereby receiving 
operational inputs from a user. 10 
[0070] The operation of the digital TV receiver appa- 
ratus 1 00 thus constructed is discussed below. 

Operation of Data Broadcasting Browser 

15 

[0071] FIG. 3 is a flow diagram showing the process 
of the data broadcasting browser 201 . When the user 
requests the digital TV receiver apparatus 100 to 
present broadcast data using the remote controller 1 32 
during viewing a TV program, the CPU 118 initiates a 20 
data broadcast receiving and replaying program (data 
broadcasting browser 201) in step S301 . 
[0072] The running data broadcasting browser 201 
acquires size information of the screen and the aspect 
ratio from the display 134 (step S302). The digital TV 25 
receiver apparatus 100, connected to the display 134 
through the serial bus, reads the screen size information 
and aspect ratio information stored in the memory of the 
display 134 through the serial bus. For example, infor- 
mation, such as "screen size = 1 9.2 inches and aspect 30 
ratio = 16:9", is thus acquired. The data broadcasting 
browser 201 stores the acquired information in the 
memory 112. 

[0073] The data broadcasting browser 201 controls 
the transport decoder 103, thereby receiving data 35 
broadcast contents transmitted in the data carousel 
method. The data broadcasting browser 201 then ac- 
quires on-screen XML data and its associated data, 
such as an image, embedded in the XML data, and store 
these pieces of data in the memory 112 (step S303). *o 
[0074] FIG. 4 shows an example of a startup screen 
of data broadcast XML data received by the digital TV 
receiver apparatus 1 00 and correspondingly associated 
with a shopping program. This example includes a com- 
ponent for <object> (object tag labeled (A) in FIG. 4) 
displaying a program on air, a component for a link but- 
ton (labeled (B) in FIG. 4) displaying detailed information 
of a product currently introduced in the program, and 
components for link buttons (labeled (C), (D), and (E) in 
FIG. 4) for switching to a general view, texture view (en- so 
iarged view), real-size view, and order screen. 
[0075] Each component is subject to style definition 
defining a display position and display format. For ex- 
ample, an object component labeled (A) in FIG. 4 is po- 
sitioned within an area extending by a width of 1 695 pix- 55 
els from a position 40 pixels rightward from the left edge 
of the screen and a height of 856 pixels from a position 
60 pixels down from the top of the screen. Each link but- 
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ton has a description of XML data which is displayed 
when the user presses the link button. 
[0076] When an image associated with the XML data 
is acquired, the data broadcasting browser 201 exe- 
cutes a script function described in OnLoad attribute of 
<body> tag in the XML data, as an initial script (step 

5304 in FIG. 3). A script is described between <script> 
and </script> in FIG. 4, and initialize function is an initial 
script in this example. 

[0077] FIG. 5 is a flow diagram of the initial script proc- 
ess described in the startup screen XML data shown in 
FIG. 4. 

[0078] In the initial script, a function is invoked to in- 
quire of the current data broadcasting browser 201 
about the working environment, in other words, to in- 
quire whether the digital TV receiver apparatus 1 00 has 
a real-size presentation feature (step S501 in FIG. 5). 
[0079] The data broadcasting browser 201 has Get- 
BrowserSupport () as a builtin API (Application Pro- 
gramlnterface) which is invoked from the script. If a 
character sequence "realsize" as an argument is given, 
the API returns a value "1" (TRUE) or "0" (FALSE) as a 
variable representing display permission attribute. 
[0080] Since the digital TV receiver apparatus 100 
and the data broadcasting browser 201 , now under dis- 
cussion, have a real-size presentation feature, the re- 
turn value of the API is TRUE (step S503 in FIG. 5). If 
this script is executed on the data broadcasting browser 
201 having no real-size presentation feature, the return 
value of the API becomes FALSE (step S504 in FIG. 5). 
[0081] In the above script, the return value of the API 
is substituted for a "visibility" attribute which decides 
whether or not to permit displaying of "a link button to 
real-size presentation". In other words, if the return val- 
ue of the API is TRUE, "visible" is substituted for the 
visibility attribute to enable displaying the link button to 
the real-size presentation. If the return value of the API 
is FALSE, "hidden" is substituted for the visibility at- 
tribute to disable displaying the link button to the real- 
size presentation. 

[0082] Subsequent to the execution of the initial 
script, the data broadcasting browser 201 renders the 
button image and text on the character and drawing im- 
age plane 109 using the graphic generator 110 (step 

5305 in FIG. 3). At the same time, the data broadcasting 
browser 201 controls the image synthesizer 111 , there- 
by synthesizing the button image and text with an image 
in the moving image plane 107 (step S306 in FIG. 3). 
[0083] FIG. 6 shows a startup screen (main screen) 
which is generated by the data broadcasting browser 
201 from the XML data. A rectangle labeled a is a mov- 
ing image window where an image of the moving image 
plane 1 07, such as a video of TV shopping, which is nor- 
mally displayed on the entire screen, is presented in a 
contracted form. On the remaining portion other than the 
moving image window, the data broadcasting browser 
201 , namely, the CPU 118 displays drawings and char- 
acters rendered on the character and drawing image 
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plane 1 09 using the graphic generator 1 1 0. A link button 
b for the real-size presentation shown in FIG. 6 appears 
because the variable of the display permission attribute 
is TRUE. 

General Presentation 

[0084] When the user selects the general presenta- 
tion button c on the main screen shown in FIG. 6 using 
the remote controller 132, general presentation display 
screen XML data is newly acquired, and the digital TV 
receiver apparatus 100 presents a screen based on the 
content of the XML data. The general presentation dis- 
plays a still image of an entire product within the display 
screen. The data broadcasting browser 201 renders re- 
ceived still image file data onto the still image plane 1 08. 
The data broadcasting browser 201 also controls the im- 
age synthesizer 111, thereby presenting a screen 
formed of a moving image window for program presen- 
tation, a still image window for still image presentation, 
and a link button for returning to the main screen shown 
in FIG. 6. 

[0085] FIGS. 7A and 7B show display examples of the 
screen. FIG. 7A shows a display example presented on 
a display 134 having a diagonal size of 29 inches, and 
FIG. 73 shows a display example presented on a dis- 
play 134 having a diagonal size of 1 9.2 inches. 

Texture Presentation 

[0086] The user may select a texture presentation 
button d on the main screen shown in FIG. 6 using the 
remote controller 132, texture presentation display 
screen XML data is newly acquired, and the digital TV 
receiver apparatus 100 presents a screen based on the 
content of the XML data. FIGS. 8A and 8B show display 
examples of the screen. FIG. 8A shows a display exam- 
ple presented on a display 134 having a diagonal size 
of 29 inches, and FIG. 8B shows a display example pre- 
sented on a display 134 having a diagonal size of 1 9.2 
inches. 

[0087] Unlike the general presentation screen, the 
texture presentation screen presents a still image which 
is captured within a close range to a product so that a 
texture of the product is seen. The data broadcasting 
browser 201 renders received still image file data to the 
still image plane 108. The data broadcasting browser 
201 also controls the image synthesizer 111, thereby 
presenting a screen formed of a moving image window 
for program presentation, a still image window for still 
image presentation, and a link button for returning to the 
main screen shown in FIG. 6. 

Real-size Presentation 

[0088] When the user selects a real-size presentation 
button b on the main screen shown in FIG. 6 using the 
remote controller 132, XML data for the real-size pres- 



entation is newly acquired, and the digital TV receiver, 
apparatus 100 presents a screen based on the content 
of the XML data. FIGS. 9 and 10 show the XML data. 
[0089] FIG. 11 is a flow diagram of an initial script 

s process described in the XML data shown in FIGS. 9 
and 10. The initial script is mainly carried out in first 
through fourth processes as described below. 
[0090] In the first process, the screen information 
such as the display size and aspect ratio of the display 

10 134 is acquired (step S1 201). 

[0091 ] In the second process, the real-size presenta- 
tion parameter calculation process is carried out using 
the XML data and the general image of the product 
(steps S1202-S1203). 

15 [0092] In the third process, an appropriate image de- 
termination process is carried out to determine whether 
an expansion rate and image size are appropriate (steps 
S1204-S1206). If it is determined that the image is not 
appropriate, the second step is carried out on another 

20 image (step S1207). If no other image is available, the 
algorithm proceeds to the fourth process. 
[0093] In the fourth process, the size of the moving 
image window is determined taking into consideration 
the final window size for real-size image presentation 

25 (step S1 208). 

[0094] The first through fourth processes will be dis- 
cussed in more detail. 

[0095] The acquisition of the screen size and aspect 
ratio in the first process is performed by invoking Get- 
30 Screenlnfo () as the API contained in the data broad- 
casting browser of the present invention from the script 
(see label (F) in FIG. 9). 

[0096] If any of four type of character sequences 
"height", "width", "size", and "ratio" as an argument is 

35 designated, the API returns a return value responsive 
to the argument. The arguments height and width re- 
spectively represent the number of pixels in a horizontal 
direction and the number of pixels in a vertical direction 
in a maximum possible size of the still image plane 108 

40 of the digital TV receiver apparatus 1 00. The argument 
size represents a diagonal size of the display 134 of the 
digital TV receiver apparatus 100. The argument ratio 
represents the aspect ratio of frame width to frame 
height of the display 134. Size "0" represents an aspect 

4 5 ratio of 4:3, and size "1" represents an aspect ratio of 
16:9. 

[0097] If a data broadcasting browser is displayed on 
the entire display screen of the digital TV receiver ap- 
paratus 100 of high-definition grade connected to the 

so display 134 having a diagonal size of 19.2 inches and 
an aspect ratio of 16:9, the return values are height = 
1080, width = 1920, size = 19.2, and ratio = 1. In the 
digital TV receiver apparatus 1 00 connected to the dis- 
play 134 having a diagonal size of 14.6 inches and an 

55 aspect ratio of 4:3, the return values are height = 480, 
width = 720, size = 14.6, and ratio = 0. 
[0098] When the script invokes the API, the data 
broadcasting browser 201 returns the values for size 
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and ratio based on the information which has been 
stored in the memory 112 at a startup, and returns the 
values for height and width based on the resolution of 
the still image plane 1 08 of the digital TV receiver appa- 
ratus 100. 

[0099] The real-size presentation is based on scaling 
information of a source image. A content producer of 
data broadcasting describes the already measured 
scaling information in the XML data. The script process 
of the data broadcasting browser 201 is thus simply to 
substitute values for variables src_h eight and src_width, 
and for variable src_dip which indicates the number of 
pixels per inch (see label (A) in FIG. 9). 
[0100] The second process of calculating the real- 
size presentation parameter is carried out as illustrated 
in a flow diagram in FIG. 12. The second process as- 
sumes three combinations of the resolution of the still 
image plane 1 08 of the digital TV receiver apparatus 1 00 
and the aspect ratio of the display 134 (see label (G) in 
FIG. 9). The present invention is not limited to these 
combinations, and calculations corresponding to vari- 
ous combinations may be performed. 

(1) Resolution of 1920 x 1080 and aspect ratio of 
16:9 

(2) Resolution of 720 x 480 and aspect ratio of 4:3 

(3) Resolution of 720 x 480 and aspect ratio of 1 6:9 

[0101] Referring to FIG. 12, height is the number of 
pixels in a height direction in the still image plane 108, 
ratio is an aspect ratio of the display 134 (16:9 = 1 , 4:3 
= 0), src_dpi is the number of pixels per inch in the 
source image, K is the number of pixels per inch in the 
display 134, Zx is a horizontal zoom ratio to be applied 
to the source image, and Zy is a vertical zoom ratio to 
be applied to the source Image. 
[0102] The value "18.37559" in the right side of the 
equation defining K represents a ratio of a diagonal line 
of the display 134 based on the aspect ratio (16:9) of 
the display 134, namely, 18.375592= 16 2 + 9 2 . Similarly, 
the value "5" in the right side of another equation defin- 
ing K represents a ratio of diagonal line of the display 
1 34 based on the aspect ratio (4:3), namely, 5 2 = 4 2 + 3 2 . 
[0103] In the combination (1), an image rendered on 
the still image plane 108 remains unchanged from that 
of the data broadcast data. The horizontal and vertical 
zoom ratios applied to the source image equal each oth- 
er, namely, Zx = Zy (step S1 304). 
[0104] In the combination (2), an image rendered on 
the still image plane 108 has a shorter width than the 
source image in the data broadcast data, and the hori- 
zontal and vertical zoom ratios applied to the source im- 
age become different, namely, Zx > Zy (step S1305). 
[0105] In the combination (3), an image rendered on 
the still image plane 108 has a longer width than the 
source image in the data broadcast data, and the hori- 
zontal and vertical zoom ratios applied to the source im- 
age become different, namely, Zx < Zy (step S1303). 



[0106] The third process of determining an appropri- 
ate Image Is performed in accordance with the flow di- 
agram shown in FIG. 13. As shown, dst_height is the 
number of pixels subsequent to an expansion process 
5 of the source image in the height direction, height is the 
number of pixels of the browser window in the height 
direction, and Zy is a vertical zoom ratio to be applied 
to the source image. 

[0107] In the third process, the image is determined 
10 as being not appropriate (step S1403) if the zoom rate 
in the height direction calculated in the second process 
is larger than a predetermined value (step S1401) and 
if the image size is larger than a predetermined value 
subsequent to the zooming of the source image in ac- 
ts cordance with the zoom rate (step S1402); otherwise, 
the image is determined as being appropriate (step 
S1404). 

[0108] In this embodiment, the image is determined 
as being not appropriate if the vertical zoom ratio is 
20 400% and if the number of pixels in a height direction 
subsequent to the zooming process of the source image 
at a zoom ratio of 400% is 70% or more of the number 
of pixels of the browser window in a height direction (see 
label (H) in FIG. 10). 

25 

Production of Texture Image from Real-size Image 

[0109] If the image is determined as being not appro- 
priate in the third process, the source image is switched 

30 from the general image to an enlarged image (texture 
image) , and the algorithm starts over again from the sec- 
ond process (step S1207 in FIG. 11). 
[0110] The adjustment of the size of the moving image 
window in the fourth process is carried out in accord - 

35 ance with a flow diagram shown in FIG. 1 4 (see label (I) 
in FIG. 10). 

[0111] As shown, dst_height is the number of pixels 
in a height direction subsequent to a zooming process 
of the source image, dst_width is the number of pixels 

40 in a width direction subsequent to the zooming process 
of the source image, height is the number of pixels of 
the browser window in a height direction, width is the 
number of pixels of the browser window in a width di- 
rection, real_height is the number of pixels of a real-size 

45 Image display window in height direction, real_width is 
the number of pixels of the real-size image display win- 
dow In a width direction, mov_height is the number of 
pixels of the moving window in a height direction, and 
mov_width is the number of pixels of moving window in 

50 a width direction. 

[0112] In the adjustment of the moving image window 
size, an image size (dst_height and dst_width) is deter- 
mined by zooming in (zooming out) the source image 
according to the zoom rate (Zx, Zy) determined in the 

55 third process. The real-size display presentation win- 
dow size (real_height, real_width) and the moving im- 
age window size (movjieight, mov_width) are deter- 
mined so that the image size falls within a constant per- 
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centage (70%, for example, as shown in FIG. 14) of the 
(browser window) screen (from step S1501 to step 
S1503). 

Display Examples 5 

[0113] FIGS . 1 5 A and 1 5B show display examples re- 
suiting from the process of the script described in the 
real-size display screen XML data. FIG. 15A shows a 
screen of the display 134 having a diagonal size of 29 10 
inches and an aspect ratio of 16:9, and FIG. 15B shows 
a screen of the display 1 34 having a diagonal size of 
19.2 inches and an aspect ratio of 16:9. 
[0114] As seen from FIGS. 15A and 15B, the image 
of the product is maintained even if the display 134 15 
changes the screen size thereof. Since the size of the 
moving image display window is adjusted to a maximum 
size in response to the window size of the real-size pres- 
entation, the TV is viewed under optimum environ- 
ments. 20 
[01 1 5] FIGS . 1 6A and 1 6B show a 1 9-inch display on 
which an image of a commercial product having a length 
of 1 meter or longer, such as a bicycle, is presented. The 
data broadcasting browser 201 of this embodiment gen- 
erates an image of interest of the product of this size, 25 
namely, a real-size image as an enlarged (texture) 
source image, as shown in FIG. 16A rather than a gen- 
eral image shown in FiG. 1 8A. Image quality of the real- 
size image is thus better than the one resulting from the 
source image shown in FIG. 1 8A. 30 
[0116] In accordance with this embodiment, realism 
felt by the user is improved because the image process- 
ing apparatus that receives data broadcast data 
presents the image of a product or commodity at the real 
size thereof during viewing a TV shopping program. Us- 35 
er confidence in the data broadcast TV shopping is thus 
promoted. 

[01 17] This embodiment allows the image of an object 
to be displayed at the reai-size regardless of specifica- 
tions of the display presenting the data broadcast pro- 40 
gram, such as the screen size and resolution. 
[0118] In accordance with this embodiment, a real- 
size presentation content is replayed only when a re- 
ceiver having a real size presentation feature receives 
a single data broadcasting content. 45 
[0119] Since window sizes in a concurrent display of 
a TV video and a product in the program associated data 
broadcasting are optimized, product presentation is car- 
ried out with the screen efficiently utilized. 
[01 20] The image quality of the real-size image is im- 50 
proved, and real-size presentation is performed without 
the need for particular user setting. 
[0121] This embodiment has been discussed in con- 
nection with the "data broadcast shopping" as a content 
of the data broadcasting. The data broadcasting content 55 
is not limited to this. The data broadcasting content may 
be the image of any object as long as the viewer wants 
to know the size of the object. For example, in a data 
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broadcasting associated with an educational program,, 
the content may be the image of animals or insects. 
[0122] The image data for presenting a real-size im- 
age is not only still image data but also moving image 
data in which commercial products, mounted on a table, 
turn integrally with the table, for example. The moving 
image data of the commercial products may be trans- 
mitted using PES (PacketizedElementaryStream) as a 
data stream method. 

[01 23] The process steps for zooming the image data 
may be stored beforehand in the receiver rather than 
being contained in the transport stream transmitted in 
the data broadcasting. In this case, a data broadcasting 
apparatus simply transmits a reference display size of 
the image data, and is freed from the responsibility to 
transmit parameters for expansion and contraction in 
accordance with the receiver. The data broadcasting ap- 
paratus thus allows receivers having a variety of speci- 
fications to display a real-size image. The present in- 
vention finds widespread use. 
[0124] In the electronic shopping using the TV receiv- 
er, users are unable to actually pick up and touch a prod- 
uct. Users are unable to try a dress on before purchasing 
it, for example, and cannot correctly picture to them- 
selves in the dress. 

[0125] To overcome such a drawback, virtual fitting 
may be contemplated by synthesizing image data rep- 
resenting the user's own image and image data of a 
commercial product on a display. 
[0126] There are a variety of TV receiver types with 
display screens thereof varied in size and resolution 
available for virtual fitting. The aspect ratio of a dis- 
played image of commercial product is different from TV 
receiver to TV receiver. The size of the image of the 
product conveyed to the user may fail to match the ac- 
tual size of the product. 

[0127] A plurality of broadcasters present a diversity 
of commodities in their programs. The products are sub- 
stantially different in size, and the product images can- 
not be synthesized with the user image. 
[0128] Electronic fitting or virtual fitting using broad- 
casting, communication, or separate storage media has 
been proposed. For example, according to Japanese 
Lald-Open Patent No. 5-336445 has proposed an image 
processing method which synthesizes and displays a 
commercial product image based on a posture and size 
of a personal appearance image when the personal ap- 
pearance image of a user and the commercial product 
image are synthesized. 

[0129] When the image processing method is applied 
to virtual fitting in an electronic shopping based on the 
data broadcasting and the TV receiver, different types 
of the product image must be broadcast concurrently in 
parallel as long as there is a possibility of being used by 
any user. This occupies a substantial portion of available 
data capacity permitted for data broadcasting, thereby 
disabling the transmission of image data of numerous 
sorts of commercial products. 
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[0130] Japanese Laid-Open Patent No. 7-21 0608 has 
proposed a sales assisting system which contracts or 
expands, synthesizes and displays product image data 
by obtaining the size of a user image from the user im- 
age or from an input unit. 

[0131] When such sales assisting system is applied 
to a virtual fitting in an electronic shopping that uses the 
data broadcasting and TV receiver, it is required to dis- 
play a diversity of types of product image data provided 
by a plurality of broadcasters. If the actual product image 
is too large with respect to the size of the user image, 
the TV screen cannot accommodate the product image. 
[0132] Since the resolution and aspect ratio of the TV 
screen presenting the product are not taken into consid- 
eration, the user may have the impression of the product 
from the TV screen, different from the actual product, if 
the aspect ratio of the shown product changes. 
[01 33] It is an object of the present invention to over- 
come this drawback. It is also another object of the 
present invention to synthesize a user image and an im- 
age derived from the data broadcast data in a correct 
scale regardless of specifications of receivers. 
[01 34] A second embodiment of the present invention 
will now be discussed in detail referring to the drawings. 
In accordance with the second embodiment, the digital 
TV receiver apparatus 100 uses a data broadcasting 
browser having a virtual fitting function, thereby synthe- 
sizing and displaying a user image and an image trans- 
mitted using data broadcasting in a correct scale. 
[0135] The digital TV receiver apparatus 100 of the 
second embodiment remains unchanged from that 
shown in the block diagram in FIG. 1 . The data broad- 
casting browser of the first embodiment has the real- 
size presentation function, while the data broadcasting 
browser of the second embodiment has a virtual fitting 
function. The discussion of the operation of the major 
portions of the digital TV receiver apparatus 1 00 is omit- 
ted here. The operation discussed below relates to the 
initiation of the data broadcasting browser based on the 
XML data transmitted using the data broadcasting and 
the virtual fitting. 

[0136] The CPU 1 1 8 in the digital TV receiver appa- 
ratus 100 operates in the same manner as already dis- 
cussed with reference to FIG. 3 except the startup 
screen XML data in the execution of the initial script in 
step S304. As a result, the rendering process of the star- 
tup screen in steps S305 and S306 results in a startup 
screen (main screen) as shown in FIG. 22. 
[0137] FIGS. 19 and 20 show an example of XML data 
of the startup screen of data broadcasting correspond- 
ingly associated with a shopping program. In this exam- 
pie, the XML data contains a component <object> (ob- 
ject tag, see label (A) in FIG. 20) for displaying a pro- 
gram on air, a link button (see label (B) in FIG. 20) for 
displaying detailed information of a product being intro- 
duced in a current program, and components (labeled 
(C), (D), and (E) in FIG. 20) for switching to a general 
view, texture view (enlarged view), real-size view, and 



order screen. 

[0138] Each component is subject to style definition 
defining a display position and display format. For ex- 
ample, each component represents that a correspond- 

5 ing object is placed in an area at a position adjusted by 
the initial script in each display screen. Specifically, an 
object component labeled (A) in FIG. 20 is placed within 
an area extending by a width of 1536 pixels from a po- 
sition 60 pixels down from the top of the screen, and by 

10 a height of 864 pixels from a position 40 pixels rightward 
from the left edge of the screen (see label (F) in FIG. 
19). Each link button has a description of XML data 
which is displayed when the user presses the link but- 
ton. 

15 [01 39] When an image associated with the XML data 
is acquired, the data broadcasting browser 201 exe- 
cutes a script function described in OnLoad attribute of 
<body> tag in the XML data, as an initial script (step 
S304 in FIG. 3). 

20 [0140] A script is described between <script> and <J 
script> in FIG. 19, and an initialize function becomes an 
initial script in this example. 

[0141] FIG. 21 is a flow diagram of the initial script 
process described in the startup screen XML data 

25 shown in FIGS. 1 9 and 20. 

[0142] In the initial script, a function is invoked to in- 
quire of the current data broadcasting browser 201 
about the working environment there, in other words, to 
inquire whether the digital TV receiver apparatus 100 

30 has a virtual fitting feature (step S501 in FIG. 21). 
[0143] The data broadcasting browser 201 has Get- 
BrowserSupport () as a builtin API (ApplicationPro- 
graminterface) which is invoked from the script (corre- 
sponding to label (G) in FIG. 19). If a character se- 

35 quence "fitting" as an argument is given, the API returns 
a value "1" (TRUE) if the digital TV receiver apparatus 
100 has the virtual fitting feature or "0" (FALSE) if the 
digital TV receiver apparatus 1 00 has no virtual fitting 
feature. 

40 [0144] Since the digital TV receiver apparatus 100 
and the data broadcasting browser 201 of the second 
embodiment have a virtual fitting feature, the return val- 
ue of the API is TRUE (step S503 in FIG. 21). If this 
script is executed on the data broadcasting browser 201 

^5 having no virtual fitting feature, the return value of the 
API becomes FALSE (step S504 in FIG. 21). In the 
above script, the return value of the API is substituted 
for a "visibility" attribute which decides whether or not to 
permit displaying of "a link button to virtual fitting page". 

50 [0145] Thescript inquires of the digital TV receiver ap- 
paratus 100 about display information of the display 
(step S505). The data broadcasting browser 201 has 
GetScreenlnfo () as the API which is invoked from the 
script (see label (H) in FIG. 19). If any of four types of 

55 character sequences "height", "width", "scale", and "ra- 
tio" as an argument is designated, the API returns a re- 
turn value responsive to the argument. The arguments 
height and width respectively represent the number of 
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pixels in a horizontal direction and the number of pixels 
in a vertical direction in a maximum possible size of the 
still image plane 1 08 of the digital TV receiver apparatus 
100. 

[01 46] The argument scale represents the number of 
pixels per unit length in a height direction of the display 
screen of the display 134. The argument ratio repre- 
sents the aspect ratio of frame width to frame height of 
the display 134. Size "0" represents an aspect ratio of 
4:3, and size " 1 " represents an aspect ratio of 1 6:9. 
[0147] In the script, the size and the position of the 
button area are adjusted depending on the number of 
pixels in the height direction, and the aspect ratio of the 
display screen of the display 134 (step S506). The 
broadcast program of the second embodiment provides 
a moving image and a still image on a display screen 
having 1920 x 1080 pixels and an aspect ratio of 16:9. 
For example, as already discussed, the moving image 
is placed in an area extending by a width of 1536 pixels 
from a position 60 pixels down from the top of the 
screen, and by a height of 864 pixels from a position 40 
pixels rightward from the left edge of the screen. If the 
acquired number of pixels of the display 134 in the 
height direction is 480, the moving image area may be 
changed to 336 pixels in width by 189 pixels in height. 
[0148] Subsequent to the execution of the initial 
script, the data broadcasting browser 201 renders the 
button image and text on the character and drawing im- 
age plane 109 using the graphic generator 110 (step 
S305 in FIG. 3). At the same time, the data broadcasting 
browser 201 controls the image synthesizer 111 , there- 
by synthesizing the button image and text with an image 
in the moving image plane 107 (step S306 in FIG. 3). 
[0149] FIG. 22 shows a display screen which is gen- 
erated by the data broadcasting browser from the XML 
data of a startup screen. As shown, a rectangular portion 
701 is a moving image window, and typically presents 
an image on the moving image plane 1 07, normally pre- 
sented on the entire display screen. The rectangular 
portion 701 thus presents an ordinary TV video in a con- 
tracted form. On the remaining portion other than the 
moving image window, the data broadcasting browser 
201, namely, the CPU 118 displays drawings and char- 
acters rendered on the character and drawing image 
plane 109 using the graphic generator 110. 
[0150] Also shown are a link button 702 which re- 
quests a link to a screen which provides detailed infor- 
mation about a commercial product introduced in the 
moving image window, for example, material, size and 
price. A link button 704 requests a shift to a screen that 
presents the general image of the product. A link button 

705 is used to perform a virtual fitting to be discussed 
later, and is displayed because a variable representing 
the display permission attribute is TRUE. A link button 

706 is used to instruct the presentation of the image of 
an object at the real size thereof. 



General Presentation 

[0151] When the user selects the Veal-size presenta- 
tion" link button 704 using the remote controller 1 32 with 

5 the main screen shown in FIG. 22 presented, the real- 
size display screen XML data is newly acquired. The dis- 
play based on the content of the XML data is presented. 
The real-size display screen presents a still image which 
is the general image captured from an entire product. 

10 [0152] The data broadcasting browser 201 renders 
the received still image file data on the still image plane 
108. The data broadcasting browser 201 also controls 
the image synthesizer 111, thereby generating a screen 
720 formed of a moving image window 722 for program 

>5 presentation and a still image window 721 for still image 
presentation as shown in FIGS. 23A and 23B, and a re- 
turn button 723 for returning to the main screen shown 
in FIG. 22. FIG. 23A shows a "sweater" as a clothing 
item on the display 134, and FIG. 23B shows an "ear- 

20 ring" as an accessory on the display 134. 

Virtual Fitting Display Page 

[0153] The user may select a "fitting" link button 705 
25 using the remote controller 1 32 on the display shown in 
FIG. 22. Virtual fitting display screen XML data is newly 
acquired, and a screen based on the content of the XML 
data is presented. FIGS. 24 through 30 show the XML 
data. 

30 [0154] FIG. 31 is a flow diagram showing an initial 
script process described in the XML data of the virtual 
fitting screen shown in FIGS. 24 through 30. The script 
process is performed as described below. 
[0155] An acquisition process for acquiring the reso- 

35 lution, aspect ratio, and scale of the display 134 is per- 
formed (see step S1002 in FIG. 31 and label (A) in FIG. 
24). With the resolution, aspect ratio, and scale of the 
display 1 34 acquired (see label (B) in FIG. 24), a product 
image is adjusted to match the display screen (step 

40 S1003). With the resolution, aspect ratio, and scale in- 
formation of the user image for virtual fitting registered 
in the digital TV receiver apparatus 100 (see label (C) 
in FIG. 25), a zooming process is performed on the user 
image so that the product image and the user image are 

45 synthesized at a correct scale. The product image and 
the user image are synthesized, thereby forming a vir- 
tual fitting image (step S1 004). 
[0156] It is then determined whether the synthesized 
virtual fitting image falls within the display area of the 

so original product image (step S1005). If it is determined 
that the virtual fitting image falls within the display area, 
the sizes and synthesis positions of the product image 
and the user image are readjusted (see step S1006 in 
FIG. 31 and label D in each of FIGS. 26 and 27). 

55 [0157] If it is determined that the virtual fitting image 
falls within the display area of the original product image, 
the sizes and synthesis positions of the product image 
and the user image are readjusted. The size and display 



10 



19 



EP1 331 812 A2 



20 



position of the product image are set (see step S1007 
in FIG. 31 and label (E) in FIG. 27), and the size and 
display position of the user image are set (see step 
S1008 in FIG. 31 and label (F) in FIG. 27). 
[0158] Each of the above steps is further discussed 5 
in detail. 

[0159] As already discussed, the acquisition process 
of the screen size and aspect ratio of the display 134 in 
step S1002 is carried out by invoking GetScreenlnfo () 
which is the API contained in the data broadcasting 10 
browser of the present embodiment from the script. 
[0160] For example, a high-definition digital TV re- 
ceiver apparatus 100 is connected to the display 134 
having a screen size of 19.2 inches and an aspect ratio 
of 16:9. If the data broadcasting browser 201 is dis- *5 
played on the entire display screen of the display 134, 
the values of height = 1080 pixels, width = 1920 pixels, 
scale =114 pixels/inch, and ratio = 1 are acquired. In 
the digital TV receiver apparatus 1 00 connected to a dis- 
play 1 34 having a screen size of 1 4.6 inches and an as- 20 
pect ratio of 4:3, the values of height = 480 pixels, width 
= 720 pixels, scale = 54.8 pixels/inch, and ratio = 0 are 
acquired. 

[0161] When the API is invoked from the script, the 
data broadcasting browser 201 provides return values 25 
for "scale" and "ratio" based on information which is read 
from the display 134 at startup through the serial bus 
and then stored in the memory 112, and provides return 
values for "height" and "width" based on the resolution 
of the still image plane 1 08 in the digital TV receiver ap- 30 
paratus 1 00. 

[0162] In broadcast program of the present embodi- 
ment, moving images and still images are broadcast on 
the assumption that the screen has 1920 x 1080 pixels 
. and an aspect ratio of 1 6:9. If the still image plane 1 08 35 
of the digital TV receiver apparatus 100 has a different 
resolution, the size and display area of the still image 
must be adjusted. Scale factors kx and ky in width and 
height directions applied to information of the product 
image and user image are calculated in step S1 002. 40 
[0163] The three combinations of resolution and as- 
pect ratio of the still image plane 108 of the digital TV 
receiver apparatus 1 00 of the present embodiment are 
contemplated. The present invention is not limited to 
these combinations, and calculations corresponding to 45 
various combinations may be performed. 

(1) Resolution of 1920 x 1080 and aspect ratio of 
16:9 

(2) Resolution of 720 x 480 and aspect ratio of 4:3 so 

(3) Resolution of 720 x 480 and aspect ratio of 1 6:9 

[01 64] In the combination (1 ), broadcast data is direct- 
ly rendered, and thus kx = ky = 1 .00. 
[0165] In the combination (2), an image rendered on ss 
the still image plane 1 08 has a shorter width. The width 
of the product image is rendered to be longer accord- 
ingly on the still image plane 108, namely, kx > ky. 



[0166] In the combination (3), an image rendered on 
the still image plane 108 has a longer width. The width 
of the product image is rendered to be shorter accord- 
ingly on the still image plane 108, namely, kx < ky. 
[0167] A content producer of data broadcasting de- 
scribes, in the XML data, information of image data set 
beforehand in step S1203 for acquiring the resolution, 
aspect ratio, and scale of the product image. The data 
broadcasting browser 201 in the script process thereof 
substitutes values for variables smpl_width, 
smpl_height, smpl_scale, smpLguidex, and 
smpLguidey. 

[0168] Here, smpLguidex, and smpLguidey are po- 
sition coordinates at which the user image overlaps the 
product image, smpLscale represents a scale where 
the user image overlaps the product image, and is ex- 
pressed in the number of pixels per unit length in a 
height direction. These values must be adjusted for the 
display 134 of the digital TV receiver apparatus 100 us- 
ing kx and ky calculated in step S1002 if the display 134 
has 720 x 480 pixels, different from the assumed reso- 
lution in the broadcasting. 

[0169] The acquisition process of the user image, res- 
olution, aspect ratio, and scale thereof in step S1004 is 
carried out by invoking, from the script, GetUserlnfo 0 
which is the API contained in the data broadcasting 
browser 201. 

[0170] If arguments "face", "chest", and "figure" are 
input, the API returns references of an image data ex- 
pressing the face, the upper half of the body of a user, 
and the entire body of the user. These pieces of the im- 
age data of the user is the image data the user has reg- 
istered beforehand in the digital TV receiver apparatus 
1 00 for virtual fitting. To perform electronic shopping, the 
image data is superimposed on product image data pro- 
vided by data broadcasting. 

[0171] The script of the data broadcasting designates 
a character sequence by selecting a type of user image 
in use for virtual fitting in accordance with a displayed 
product. The resolution, aspect ratio, and scale of the 
user image are associated with the user Image, and are 
obtained by referencing the user image. Like the scale 
of the product image, the scale of the user image is rep- 
resented in the number of pixels per unit length in a 
height direction. 

[01 72] Here, "factor" is a scale factor of contraction or 
expansion of the user image. The scale factor is used 
to set the product image and the user image in the same 
contraction or expansion rate in a synthesis process, 
and is used to contract or expand the user image to ob- 
tain a dimensionally correct synthesized image. The 
scale factor is obtained by dividing the "scale" of the * 
product image by the "scale" of the user image. Using 
the scale "factor", usr_width, usr_height, usr_guidex, 
and usr_guidey are adjusted. Since the aspect ratio of 
the user image is already adjusted to meet the display 
conditions of the display 134 in use, no further adjust- 
ment is required. 
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[0173] FIG. 32 shows a flow diagram of a readjust- 
ment process of the size and synthesis position of the 
product image and the user image as necessary so that 
the virtual fitting image falls within the display area of 
the original product image (steps S1005 and S1006). 
[0174] An overextended portion of the user image be- 
yond the product image in ± X directions is determined 
(step S1101). The product image contains guide coor- 
dinates (guidex and guidey) serving as a guide when 
the user image is synthesized (displayed in superimpo- 
sition). The guide coordinates are relative coordinates 
having the origin thereof at the top left corner of each 
user image. If the user image is displayed in superim- 
position on the product image so that their coordinates 
agree with each other, a synthesized image for virtual 
fitting is aligned in a correct positional relationship. 
[01 75] The displaying of the synthesized image is per- 
formed in the procedure described in the body of the 
script. In step S1101 , "offsetx" represents an amount of 
overextension of the user image in the -X direction from 
the product image when the two images are superim- 
posed in accordance with the position provided by the 
guide coordinates, "remainx" represents an amount of 
overextension of the user image in the X direction from 
the product image, and "factorx" representing a contrac- 
tion or expansion rate in the X direction is calculated so 
that the display area of the synthesized image falls with- 
in the display area of the original product image. 
[0176] In step S1102, an overextended portion of the 
user image beyond the product image in ± Y directions 
is determined. Similarly, the scale factor factory" in the 
Y direction is calculated from "offsety" and "remainy". 
[01 77] The process for determining a scale factor "re- 
size" for contracting the product image and user image 
in (steps S1 1 03-S1 1 05) is selecting a value "1 " requiring 
neither contraction nor expansion and the smallest val- 
ue of "factorx" and -factory". In other words, if no over- 
extension occurs, no contraction is required. A contrac- 
tion rate in the direction of larger overextension is adopt- 
ed. If no contraction is required with resize < 1 (step 

51106) , the product image data and user image data 
are contracted, and the guide coordinates, and informa- 
tion of amounts of position adjustment "offsetx" and "off- 
sety" of the product image required in response to the 
overextension of the user image are updated (step 

51107) . 

Display Examples 

[0178] Display examples of the data broadcasting 
browser 201 and the script process described in the vir- 
tual fitting screen XML data are discussed. 
[0179] Referring to FIGS. 34A-34C, clothing "sweat- 
er" as a product is tried on. In this example, the user 
image of the "face" shown in FIG. 33B is registered in 
the digital TV receiver apparatus 100, and the product 
image shown in FIG. 33A is broadcast. 
[0180] FIG. 34A shows a fitting image which is pre- 



sented on the display 1 34 of the digital TV receiver ap-. 
paratus 100. The display 134 has 1920 x 1080 pixels, 
an aspect ratio of 1 6:9, and a diagonal size of 1 9.2 inch- 
es. FIG. 34B shows a fitting image which is presented 

s on the display 1 34 of the digital TV receiver apparatus 
100. The display 134 has 720 x 480 pixels, an aspect 
ratio of 16:9, and a diagonal size of 14.6 inches. FIG. 
34C shows a fitting image which is presented on the dis- 
play 134 of the digital TV receiver apparatus 100. The 

10 display 134 has 720 x 480 pixels, an aspect ratio of 4: 
3, and a diagonal size of 14.6 inches. 
[0181] FIG. 34A shows a still image window 731 for 
presenting a synthesized image, a moving image win- 
dow 732 for program presentation, a shift button 733 for 

15 shifting a synthesized image within the still image win- 
dow 731, and a return button 734 for returning to the 
main screen shown in FIG. 7 (The same is true of FIGS. 
34B and 34C). 

[0182] In any of the displays 134, the user image is 

20 contracted or expanded to a correct size with respect to 
the product image, and synthesized and displayed. In 
the displays 134 having specifications different from 
specifications assumed in the data broadcasting, re- 
spectively shown in FIGS. 34B and 34C, the user image 

25 and product image with the correct sizes and correct as- 
pect ratios maintained are synthesized and displayed, 
although the layout of the on-screen images are slightly 
different from that shown in FIG. 34A. A correct image 
of the product in the virtual fitting is thus conveyed using 

30 the data broadcasting. 

[0183] Referring to FIGS. 35A-35C, "earrings" as an 
accessory are tried on. FIG. 35A shows a fitting image 
which is presented on the display 134 of the digital TV 
receiver apparatus 100. The display 134 has 1920 x 

35 1 080 pixels, an aspect ratio of 1 6:9 , and a diagonal size 
of 1 9.2 inches. FIG. 35B shows a fitting image which is 
presented on the display 134 of the digital TV receiver 
apparatus 100. The display 134 has 720 x 480 pixels, 
an aspect ratio of 1 6:9, and a diagonal size of 1 4.6 inch- 

40 es. FIG. 34C shows a fitting image which is presented 
on the display 134 of the digital TV receiver apparatus 
100. The display 134 has 720 x 480 pixels, an aspect 
ratio of 4:3, and a diagonal size of 14.6 inches. 
[0184] As shown in FIGS. 35A-35C, the user image 

4 5 registered in the digital TV receiver apparatus 1 00 is ap- 
propriately contracted or expanded, and displayed with 
respect to the product image data although the product 
(the earrings) is substantially different in size from the 
product (the sweater) shown in FIG. 33A. A correct prod- 

50 uct image is thus conveyed to the viewer. 

[0185] In this embodiment, the scale of the broadcast 
image data is adjusted to match the scale of the display 
screen of the receiver. The scale factor is obtained by 
dividing the scale of the broadcast image data by the 

55 scale of the user image data which is set to be coincident 
with the scale of the display screen and stored. The user 
image data is contracted or expanded according to the 
scale factor, and synthesized with the broadcast image 
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data. The synthesized image data is then displayed on 
the display screen. 

[0186] In the data broadcasting service such as an 
electronic (data broadcasting) shopping in accordance 
with this embodiment, the user image registered in the 5 
receiver and the product image are contracted or ex- 
panded at a correct scale factor even in environments 
where receivers with displays having different specifica- 
tions coexist. A virtual fitting is thus performed on the 
display screen of the TV receiver. 10 
[0187] The electronic shopping, in which the viewer is 
unable to touch the product directly, conveys a correct 
product image to the viewer, and allows the viewer to 
try the product on virtually with an easy operation. The 
viewer thus feels "realism", and user confidence in the *5 
data broadcast TV shopping is promoted. 
[0188) In the present embodiment, the data broad- 
casting shopping is performed in the data-broadcasting 
program. The virtual fitting is based on the face of the 
user in the above discussion. But the chest, or the entire 20 
body of the user may be used in the virtual fitting. The 
image of the fingers, the hand, and the arm of the user 
may be used to try an accessory or a wristwatch on. A 
building may be tried using the entire body or furniture 
as the user image. Furthermore, an indoor view may be 25 
used as the user image to select virtually furniture. 
[0189] The present invention is not limited to the vir- 
tual fitting in the electronic shopping. The image data 
held for individual users with correct dimensions main- 
tained and the broadcast data are synthesized based 30 
on the data broadcasting that is available to wide public. 
[0190] The synthesis process is not limited to the still 
image. A content may be formed of broadcast moving 
images and the user image. To handle the still image, 
the data carousel transmission method Is preferable. 35 
The moving image is preferably transmitted using a PES 
(PacketizedElementaryStream) method. 
[0191] The process for contracting or expanding the 
image data of a product may be stored in the receiver 
rather than being contained in the transport stream *o 
transmitted in the data broadcasting. In this case, a data 
broadcasting apparatus simply transmits the scale of 
the image data as data for contraction and expansion, 
and is freed from the responsibility to transmit process 
information for expansion and contraction in accord- 45 
ance. The arrangement reduces the amount of trans- 
mission data and allows multimedia data to be addition- 
ally transmitted. 

[0192] The present invention is not limited to the 
above-referenced embodiments, and finds use in any of so 
the functions to be discussed in claims or functions per- 
formed by the above- referenced embodiments. 
[0193] For example, the software configuration and 
the hardware configuration in the above-referenced em- 
bodiments are interchangeable with each other. The el- 55 
ements of the above-referenced embodiments may be 
combined as necessary. The construction specified in 
each of the claims or a part or whole of each of the 
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above- referenced embodiments may form a single ap- 
paratus, or may be combined with another apparatus, 
or may be an element of an apparatus. 
[0194] In accordance with the present invention, the 
image data obtained from data broadcasting is dis- 
played in a form reflecting the real size of an object re- 
gardless of the specifications of the receiver. The user 
image and the image from the data broadcasting are 
synthesized and displayed at a correct scale factor. The 
user thus correctly judges whether or not to purchase 
the product in the data broadcasting shopping. 
[0195] While the present invention has been de- 
scribed with reference to what are presently considered 
to be the preferred embodiments, it is to be understood 
that the invention is not limited to the disclosed embod- 
iments. To the contrary, the invention is intended to cov- 
er various modifications and equivalent arrangements 
included within the spirit and scope of the appended 
claims. The scope of the following claims is to be ac- 
corded the broadest interpretation so as to encompass 
all such modifications and equivalent structures and 
functions. 



Claims 

1. An apparatus for receiving broadcast data, com- 
prising: 

receiving means for receiving image data of the 
image of an object and size information relating 
to the object, transmitted through data broad- 
casting; 

means for generating screen information relat- 
ing to a display screen of display means; 
scaling means for determining a scale factor of 
the object image based said screen information 
and the size information received by the receiv- 
ing means so that the object image is displayed 
on the display means at a predetermined size; 
and 

display control means for controlling the display 
means to display the object image expanded or 
contracted at the scale factor determined by the 
scaling means. 

2. An apparatus according to claim 1, wherein the 
screen information includes the size and aspect ra- 
tio of the display screen. 

3. An apparatus according to claim 1 , wherein the ob- 
ject image is an image captured from the object, and 
the predetermined size is the real size of the object. 

4. An apparatus according to claim 1 , wherein the re- 
ceiving means receives a plurality of types of object 
images of the object. 
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5. An apparatus according to claim 4, wherein the plu- 
rality of types of object images include at least one 
of a general image of the entire object and a zoom 
image which is captured by zooming a portion of the 
object. 

6. An apparatus according to claim 4, wherein the 
scaling means scales a second object image differ- 
ent in type from a first object image among the plu- 
rality of types of object images when the determined 
scale factor of the first object image exceeds a pre- 
determined value. 

7. An apparatus according to claim 4, wherein the 
scaling means scales a second object image differ- 
ent in type from a first object image among the plu- 
rality of types of object images when the first object 
image is sized to occupy the area of the display 
screen at a predetermined percentage or more. 

8. An apparatus according to claim 1 , wherein the re- 
ceiving means receives a broadcast program 
formed of moving image data, while also receiving 
an object image, as still image data, in the program 
transmitted through the data broadcasting. 

9. An apparatus according to claim 8, wherein the stilt 
image data is broadcast using the data carousel 
method. 

10. An apparatus according to claim 8, wherein the dis- 
play control means determines a size of a moving 
image of the broadcast program based on the size 
of the scaled object image scaled by the scaling 
means so that the moving image of the broadcast 
program and the object image are displayed on the 
same display screen. 



1 2. A method according to claim 1 1 , wherein the screen . 
information includes a size and an aspect ratio of 
the display screen. 

5 1 3. A method according to claim 1 1 , wherein the object 
image is an image captured from the object, and the 
predetermined size is the real size of the object. 

14. A method according to claim 11, wherein the receiv- 
10 ing step comprises receiving a plurality of types of 

object images of the object. 

1 5. A method according to claim 1 4, wherein the plural- 
ity of types of object images include at least one of 

is a general image of the entire object and a zoom im- 
age which is captured by zooming a portion of the 
object. 

1 6. A method according to claim 1 4, wherein the scaling 
20 step comprises scaling a second object image dif- 
ferent in type from a first object image among the 
plurality of types of object images when the deter- 
mined scale factor of the first object image exceeds 
a predetermined value. 

25 

17. A method according toclaim 14, wherein the scaling 
step comprises scaling a second object image dif- 
ferent in type from a first object image among the 
plurality of tvpes of object images when the first ob- 

30 ject image is sized to occupy the area of the display 
screen at a predetermined percentage or more. 

1 8. A method according to claim 1 1 , wherein the receiv- 
ing step comprises receiving a broadcast program 

35 formed of moving image data, while also receiving 
an object image, as still image data, in the program 
transmitted through the data broadcasting. 



11 . A method for displaying a data broadcast program, 
comprising: 

a receiving step of receiving image data of the 
image of an object and size information relating 
to the object, transmitted through data broad- 
casting; 

an input step of inputting screen information re- 
lating to a display screen of display means; 
a scaling step of determining a scale factor of 
the object image based on the screen informa- 
tion input in the input step and the size informa- 
tion received in the receiving step so that the 
object image is displayed on the display means 
at a predetermined size; and 
a display control step for controlling the display 
means to display the object image expanded or 
contracted at the scale factor determined in the 
scaling step. 



1 9. A method according to claim 1 8, wherein the display 
control step comprises determining a size of a mov- 
ing image of the broadcast program based on the 
size of the object image scaled in the scaling step 
so that the moving image and the object image are 
displayed on the same display screen. 

20. A computer program for displaying a data broadcast 
program, comprising: 

a receiving step of receiving image data of the 
image of an object and size information relating 
to the object, transmitted through data broad- 
casting; 

an input step of inputting screen information re- 
lating to a display screen of display means; 
a scaling step of determining a scale factor of 
the object image based on the screen informa- 
tion input in the input step and the size informa- 
tion received in the receiving step so that the 
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object image is displayed on the display means 
at a predetermined size; and 
a display control step for controlling the display 
means to display the object image expanded or 
contracted at the scale factor determined in the $ 
scaling step. 

21 . A computer program according to claim 20, wherein 
the computer program is transmitted as data broad- 
cast data. 10 

22. An apparatus for receiving broadcast data, com- 
prising: 



image data at a scale factor that is obtained by di- . 
vidingthe scale of the adjusted broadcast image da- 
ta by the scale of the user image data that is already 
set to coincide with the scale of the display screen 
of the display means. 

28. An apparatus according to claim 22, wherein the 
scaling means performs a scaling process so that 
the synthesized image data of the broadcast image 
data and the user image data falls within a display 
area of the original broadcast image data when the 
synthesized image data overextends the display ar- 
ea of the original broadcast image data. 



receiving means for receiving broadcast image '5 
data transmitted through data broadcasting; 
display control means for synthesizing the 
broadcast image data received by the receiving 
means and user image data registered before- 
hand and for causing display means to display 20 
the synthesized data; and 
scaling means for scaling the broadcast image 
data and the user image data so that the broad- 
cast image data, the user image data, and the 
display screen of the display means coincide 25 
with each other in scale, 

wherein the display control means synthesiz- 
es the broadcast image data and the user image 
data scaled by the scaling means, and causes the 30 
display means to display the synthesized data. 

23. An apparatus according to claim 22, wherein the 
broadcast image data contains scaling information 

to indicate a scale of the image of an object repre- 35 
sented by the broadcast image data. 

24. An apparatus according to claim 22, wherein the us- 
er image data contains scaling information to indi- 
cate a scale of the image of an object represented *o 
by the user image data. 

25. An apparatus according to claim 22, wherein the 
broadcast image data to be synthesized with the us- 
er image data contains position information which « 
is used to synthesize the user image data onto the 
broadcast image data. 



29. An apparatus according to claim 22, wherein a plu- 
rality of pieces of user image data are registered, 
and 

wherein the broadcast image data contains 
designating information that designates user image 
data corresponding to an object represented by the 
broadcast image data. 

30. A method for displaying a data broadcast program, 
comprising: 

a receiving step of receiving broadcast image 
data transmitted through data broadcasting; 
a display control step of synthesizing the broad- 
cast image data received in the receiving step 
and user image data registered beforehand 
and causing display means to display the syn- 
thesized data; and 

a scaling step of scaling the broadcast image 
data and the user image data so that the broad- 
cast image data, the user image data, and the 
display screen of the display means coincide 
with each other in scale, 

wherein the display control step comprises 
synthesizing the broadcast image data and the user 
image data scaled in the scaling step and causing 
the display means to display the synthesized data. 

31 . A method according to claim 30, wherein the broad- 
cast image data contains scaling information to in- 
dicate a scale of the image of an object represented 
by the broadcast image data. 



26. An apparatus according to claim 22, wherein the us- 
er image data contains position information which 
is used to synthesize the user image data onto the 
broadcast image data. 



32. A method according to claim 30, wherein the user 
so image data contains scaling information to indicate 
a scale of the image of an object represented by the 
user image data. 



27. An apparatus according to claim 22, wherein the 
scaling means adjusts the scale of the broadcast ss 
image data to cause the scale of the broadcast im- 
age data to coincide with a scale of the display 
screen of the display means, and scales the user 



33. A method according to claim 30, wherein the broad- 
cast image data to be synthesized with the user im- 
age data contains position information which is 
used to synthesize the user image data onto the 
broadcast image data. 
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34. A method according to claim 30, wherein the user 
image data contains position information which is 
used to synthesize the user image data onto the 
broadcast image data. 

5 

35. A method according to claim 30, wherein the scaling 
step includes adjusting the scale of the broadcast 
image data to cause the scale of the broadcast im- 
age data to coincide with a scale of the display 
screen of the display means, and scales the user io 
image data at a scale factor that is obtained by di- 
viding the scale of the adjusted broadcast image da- 
ta by the scale of the user image data that is already 

set to coincide with the scale of the display screen 
of the display means. 15 

36. A method according to claim 30, wherein the scaling 
step comprises performing a scaling process so 
that the synthesized image data of the broadcast 
image data and the user image data falls within a 20 
display area of the original broadcast image data 
when the synthesized image data overextends the 
display area of the original broadcast image data. 

37. A method according to claim 30, wherein a plurality 25 
of pieces of user image data are registered, and 

wherein the broadcast image data contains 
designating information that designates user image 
data corresponding to an object represented by the 
broadcast image data. . 30 

38. A computer program for displaying a data broad- 
casting program, comprising: 

a receiving step of receiving broadcast image 35 
data transmitted through data broadcasting; 
a display control step of synthesizing the broad- 
cast image data received in the receiving step 
and user image data registered beforehand 
and causing display means to display the syn- 40 
thesized data; and 

a scaling step of scaling the broadcast image 
data and the user image data so that the broad- 
cast image data, the user image data, and the 
display screen of the display means coincide 
with each other in scale, 

wherein the display control step comprises 
synthesizing the broadcast image data and the user 
image data scaled in the scaling step and causing 50 
the display means to display the synthesized data. 

39. A computer program according to claim 38, wherein 
the computer program is transmitted as data broad- 
cast data. 55 
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FIG. 3 

( START ) 



START BROADCAST DATA RECEIVING AND 
REPLAYING PROGRAM IN RESPONSE TO DATA K S301 
RECEIVING INSTRUCTION FROM USER 



ACQUIRE AND STORE DISPLAY SIZE, AND 
RATIO OF FRAME WIDTH TO FRAME HEIGHT 



RECEIVE AND STORE BROADCAST DATA 
IN MEMORY 



-S302 



S303 



PERFORM INITIAL SCRIPTS ON PAGE 



S304 



RENDER PAGE INFORMATION 



-S305 



SYNTHESIZE MOVING IMAGES 



S306 



( ) 
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FIG. 4 



<!DOCTYPE dbc SYSTEM w httpy/\wwwjcxx.cojp/dbc.dt<T> 
<head> 

<titleData DATA SHOPPING</titie> 

(script language="JabaScript"> 
var realenable = false; 

function initialized 
realenable = Browser.GetBrowserSupport{ realsize ) 
var a=documentgetElementByWfreaLbtn"); 

if(rea!enab!e=trueX 
a.style.visibility^visible"; 

i 

else! 

a.style.visibility^'hidden 1 '; 

i 

i 

</script> 
</head> 

<body onLoad=' f initializeO*> m 
<div style=>sition:absolute; fontrsize:24; top:20; Ieffc40; width:1696; height:32 visibifity^isible; > 

</div> 

<div style= Vsitiorcabsolute; top:60, Ieft40; width:1 696; height856 visibility*i$ib!e; > 1 , . v 

<object id=1ive>pe=\ideo/mpeg2* data="00"></object> J 

</div> _ . 

<div style=>sition:absolute; top:100; leffcl 760; width:1 36 height64 visibOitytfsible; > | ^ 

<a hre^specMm") ' 
<object id=*btnr,type=*image/ipeg* data=*btn.specjpg*></object> 

</a> 

</div> „ . 

<div style=Vsition:absolute; topSQO; !eft1760; widlh:136 height64 visibility^isible; > I (q) 

<a href= w imgalLhtm"> 

<object i*=*btn2^ype="image/jpeg" data="btnj*lljpg"></object> 
</a> 

</div> „ . 

<div style="posKion:absolute; top:300; leffcl 760; width:! 36 height64 visibility*isible; > I (p) 

<a href=' , img.zoomiitm*> 
<object id^b^type^mage/W data='btn.zoomjpg"></object> 

</a> 

</div> m . 

<div style=>sition:absolute; top:400; leffcl 760; width:1 36 height64; > 1(g) 

<a href="img_reali»tfn J '> 

<object id="btn4^type="image/jpeg" daU="btn.realjpg"></object> 
</a> 
</div> 
</body> 
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FIG. 5 



( START OF INITIAL SCRIPT ) 



INQUIRE OF BROWSER ABOUT POSSIBILITY 
OF REAL SIZE PRESENTATION 



S501 




S503 



REAL-SIZE BUTTON 
DISPLAY NOT PERMITTED = FALSE 



REAL-SIZE BUTTON 
DISPLAY PERMITTED = TRUE 



FIG. 6 



GRANDFATHER'S TABLE CLOCK 




( DETAIL ) 




MOVING IMAGE AREA 


( GENERAL > 


a 


( TEXTURE > 




(REAL size) 
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FIG. 7A 




IMAGE 
DISPLAY 
AREA 



(RETURN) 



GENERAL IMAGE DISPLAY AREA 



FIG. 7B 




GENERAL IMAGE DISPLAY AREA 
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FIG. 8A 





IMAGE 
DISPLAY 
AREA 




I (return) 



TEXTURE IMAGE DISPLAY AREA 



FIG. 8B 





IMAGE 
DISPLAY 
AREA 




/ (RETURN) 



TEXTURE IMAGE DISPLAY AREA 
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FIG. 9 



<!DOCTYPE dbc SYSTEM "http://www.xxx.cojp/dbc.Acr> 
<head> 

<titleData DATA SHOPPING</title> 

<script language^ JavaScript") 
var height,width,size f ratio; 
var dst_heightdst_width; 
var rw.height, rw.width; 
var mov_height mov_width; 
var P, K, Zx. Zy. 

var srcfite = 

var src.height, src.width, sro.dpi; 

function inrtiallzeOt 

src.file = "camera_all jpg" 
src}jeight=1024; 
src.width = 768; 
src_dpi= 100; 



! 



(A) 



getparameteK); 
calculateO; 

fK checkO=tfalse )l 

src.file = "camera^zoomjpg" 
src.height = 1024; 
src.width = 768; 
src_dpi= 300; 
calculateO; 



(B) 
(O 



■(D) 



) 

adjustwindowO; 



}(B 



) 

function getparameterOI 0 

height = Brows er.GetScreeninfo(^ height r, 
width = Browser.GetScreenlnfbCwidto''); 
size = Browser.GetScreenlnfoC^size*); 
ratio = Browser.GetScreenlnfoCratio"); 

} 

function calculateO! 
iftheight==480)l 
rf(ratio=1)( 

K=height(=1 80)/(size* (9/1 8.37559)) 

Zy=K/src_dpi; 

Zx=Zy/1.185; 

} 



(F) 



K=height(=480)/(size*(3/5)) 

Zy=K/src_cfc>i; 

Zx=Zy/0.8B8; 



else( 



K=height(=1 080)/(size* (9/1 8.37559)) 
Zy=k/src_dpi; 
Zx=Zy; 



(G) 
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FIG. 10 




return false; 



(H) 



else{ 



return true; 



function aa^ustwindowOI 
dst.height = srcheight* Zy, 
dst width = src_width*2x; 
if( dst height>height*70/1Q0X 

rw height = height* 70/1 00; 

rw width = width* 70/100; 

mov height = height* 30/1 00; I (\) 

mov width = width* 30/1 00; w 

} 

else{ 

rw.height = dst.height 
rw.width = dstwidth; 
mov.height = height-dst.height; 
mov.width = width-dst_width; 

documentall.RealImg.data =sec_file; 

documentall.Reallmg.styte.width =dst_width; 

documentall.RealImg.stylei)eight=dst.height; 

documentai!.RealImg.data =sec_file; 

documentalI.RealImg.style.top = 0; 

documentall.RealDiv.style.Ieft = 0; 

documentall.RealDiv.style.width = rw.width; 

documentall.Rea!Div.style.height = rw_height 

documentalLLtveOiv^tv1e.top = rw.width; 

documentall.LiveDiv.style.left = 0; 

documentall.UveDiv.style.width = mov.width; 

documenta!I.L?veDiv.sty1e.height = movjieight 

documentall.bve.style.top = rw.width: 

documentall.Live.style.left = 0; 

documentall.Live.style.width = mov.width; 

documentall.Live.styie.height = mov.height) 
</script> 
</head> 

<body onLoad=*initiaiizeO Jr > 

<div id^RealDiv" style="positionabsolute; visibilrty:visible;"> 

<object ia^'Reallmg'.type^image/jpeg'X/objecO 
</div> 

<div id="LiveDiv" style=~position:absolute; visibilrtyrvisible; > 
<object id="IJve'^type=*video/mpeg2 , ' data="00"></object> 
</div> 

<div style="position:absolute; top:800; Ieft1760; width:136 height64 

visibility:visible;"> 
<a href="startup.htnrT> m 

<object id="btnback2" r t type="image/jpeg ,,, data= btn.backjpg ></object> 



</a> 
</div> 
</body> 
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FIG. 1 1 



( START OF INITIAL SCRIPT ) 



ACQUIRE SCREEN SIZE OF DISPLAY, 
AND ASPECT RATIO OF WIDTH TO 
HEIGHT SCREEN 



SOURCE IMAGE = GENERAL IMAGE 



CALCULATE PARAMETER OF 
REAL-SIZE DISPLAY 



S1204 



DETERMINE PARAMETER 



S1201 



S1202 



S1203 




SI 205 




APPROPRIATE ? 
YES 



S1207 



MODIFY SOURCE IMAGE 



CALCULATE WINDOW SIZE OF 
MOVING IMAGE 



S1208 



( END *) 
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FIG. 12 

height = NUMBER OF PIXELS IN HEIGHT DIRECTION IN STILL IMAGE PLANE 

ratio = ASPECT RATIO OF DISPLAY (16:9 = 1,4:3 = 0) 

src dpi = NUMBER OF PIXELS PER INCH IN SOURCE IMAGE 

K = I NUMBER OF PIXELS PER INCH IN DISPLAY 

Zx = HORIZONTAL ZOOM RATIO TO BE APPLIED TO SOURCE IMAGE 

Zy = VERTICAL ZOOM RATIO TO BE APPLIED TO SOURCE IMAGE 



( START OF CALCULATION ) 
-S1301 



1920X1080(16: 9) „ 




S1302 
YES 



K = height(= 1080) 
/(size* (9/18.37559)) 
Zy = Zx = k/src_dpi 



S1304 



720X480(4 :3) u 



n 720X480(1 6: 9) 



K = height(= 480) 
/(size* (9/1 8.37559)) 
Zy = k/src_dpi 
Zx = Zy/1.185 



K = height(= 480) 
/(size* (3/5)) 
Zy = k/src_dpi 
Zx = Zy/0.888 



S1305 



S1303 



( ) 
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FIG. 13 

dst height = NUMBER OF PIXELS SUBSEQUENT TO EXPANSION PROCESS 

" OF SOURCE IMAGE IN HEIGHT DIRECTION 

height = NUMBER OF PIXELS OF BROWSER WINDOW IN HEIGHT DIRECTION 
Zy = VERTICAL ZOOM RATIO TO BE APPLIED TO SOURCE IMAGE 



( APPROPRIATENESS DETERMINATION PROCESS ) 

S1401 
Zy LARGER THAN 400% ? 




DETERMINE IMAGE AS BEING 
NOT APPROPRIATE 



DETERMINE IMAGE AS BEING 
APPROPRIATE 



1 

( END ) 
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FIG. 14 

dst height = NUMBER OF PIXELS IN HEIGHT DIRECTION SUBSEQUENT TO 

EXPANSION PROCESS OF SOURCE IMAGE 
dst width = NUMBER OF PIXELS IN WIDTH DIRECTION SUBSEQUENT TO 

EXPANSION PROCESS OF SOURCE IMAGE 
height = NUMBER OF PIXELS OF BROWSER WINDOW IN HEIGHT DIRECTION 
width = NUMBER OF PIXELS OF BROWSER WINDOW IN WIDTH DIRECTION 
real height = NUMBER OF PIXELS OF REAL-SIZE IMAGE DISPLAY WINDOW 

IN HEIGHT DIRECTION 
real width = NUMBER OF PIXELS OF REAL-SIZE IMAGE DISPLAY WINDOW 

IN WIDTH DIRECTION 
mov height = NUMBER OF PIXELS OF MOVING WINDOW IN HEIGHT DIRECTION 
movwidth = NUMBER OF PIXELS OF MOVING WINDOW IN WIDTH DIRECTION 



( MOVING WINDOW ADJUSTMENT) 



S1501 




real.height = dst.height 
real.width = dstwidth 
mov.height - 
height-dstheight 

movwidth = 
width-dst width 



S1503 



SET real height AND real width 
TO BE 70% OF height AND width, 

RESPECTIVELY 
SET mov height AND mov_width 
TO BE 30% OF height AND width, 
RESPECTIVELY 



< 

( "> ) 
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1 



FIG. 1 5A 





MOVING 
DISPLAY AREA 




REAL-SIZE 


DISPLAY AREA 


(RETURN) 



FIG. 1 5B 





.©1 




MOVING 
DISPLAY AREA 






(return) 



j 

FIG. 16A FIG. 16B 



A 


MOVING 
DISPLAY AREA 




I (RETURN) 
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FIG. 18A 
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FIG. 1 9 



<!DOCTYPE dbc SYSTEM "http://wwwJooc.cojp/dbc.cftd"> 
<head> 

<titleData DATA SHOPPING</title> 

<script language="JavaScript"> 
var enableToFttting = false; 

function initialize^ m *) 

var enableToFitting = BrowserGetBrowserSupportCfitting") ^ G 
a=documentgetElementBy1d("btn_3* r ); 
if( enableToFitting=true){ 

a.style.visibility^ visible"; 
J else { 

a.style.visibility="hidden' r ; 

var scrn.height = Browser. GetScreenlnf ©("height"); 
documentall.divjive.style 

= "position:absolute; top:60; Ieft40; width: 1536; height864 visil 
documeritall.divbtnl. style 

= "position:absolute; top:60; left! 61 6; width:264; height162 visibility^sible; 
documentall.divJ>tn2.style m 

= "position:absolute; top:316; Ieft1616; width:264; height162 visibility:visrble; 
documentall.drv_btn3.style 

= "posrtion:absolute; top:572; Ieft1616; width:264; height162f 
documental I.drv_btn4 .style 

= "positionabsolute; top:828; left*1616; width:264; height162 visibilityzvisiblef 
if ( scrn.hight = 480) { 

document.all.divJive.style.width = 336; 

documentall.divJive.height = 1 89; 

documentall.div_btn1Jeft = 416; 
documentall.div_btn1 .height = 70; 
documentall.div.btn2.top = 150; 
documentall.div.btn2.left = 416; 
documentall.div_btn2.height = 70; 
documentall.div_btn3.top = 240; 
documentall.div_btn3.left = 416; 
documentall.div_btn3.height = 70; 
documentall.div_btn4.top = 310; 
documentall.div.btn4.left = 41 6; 
documentall.div_btn4.height = 70; 




33 



EP 1 331 812 A2 



FIG. 20 



) 

} 

</script> 
</head> 

<body onLoad="initialize(r> 

<div sty!e="posltion:absolute; font-size:24; topSO; Ieft40; width:180; height40; visibi!ity*isible; > 
</div> 



<div id = "divjive^ \ A 

<otiect id='live',type="video/mpeg2' data="00 ></oyect> rA 

Vdiv> 

!div id = "divjrtnl *> 

<a href="spec.htiTr> m -\ 

<oy ect »d="btnr,type="image/jpeg* data=~btn.specjpg"></object> L Q 
<Va> J 



</a> 
</div> 

<div id = "divj)tn2~> 
<a href^imagalLhtnO 



<a hret= imagaiuitm ^ >w 
<object id="btn2" > type="image/jpeg" data=~btnjmgall jpg"></object> L q 
</a> J 



</a> 
</div> 

<div id = ~div_btn3~> 
<a href="fitting.htm"> 



^a nret= mting.nun / -\ 
<object id="btn3^type=*'image/jpeg'" data="btn_fittingjpg"></object> I Q 
</a> J 



</dlv> 

<div id = "'div_btn4 /F > 
<a href="order.htnT> 



\a nreT= oraer.nun / a 
<object id="btn4" r type="image/jpeg' r data="btn.orderjpg"></object> I £ 
</a> J 



</div> 
</body> 
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700 



FIG. 22 

701 702 



SOFT AND LIGHT SWEATER 




-f- 

( DETAIL Y 



GENERAL 



( FITTING 
( REAL-SIZE 



703 
704 
705 
706 



720 

_Z_ 



FIG. 23A 

721 



722 

) 



SOFT AND LIGHT SWEATER, 





( RETURN > --723 



FIG. 23B 

721 722 
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FIG. 24 



<!DOCTYPE dbc SYSTEM "http://www.xxx.cojp/dbc.dtd"> 
<head> 

<titleData DATA FITTING</title> 

<script language="JavaScript"> 

var scrnjieight, scrn.width, scrn.ratio, scrn.scale; 

var smpLwidth, smpLheight, smpl_scale; 

var smpLguidex, smpl _guidey, smpljeft, smpLtop; 

var usr_source f usr_height usr.width, usr.scale, usr _guidex; 

var uscguidex, usr_guidey, usrjeft, usr.top; 

var enableToFitting, kx f ky, offsetx, offsety; 

function initializeO { 

var enableToFitting = Browser.getBrowserSupport("fitting"); 
if ( enableToRtting = TRUE ) { 

getScreenlnformationO; 

getSamplelnformationO; 
getUserlnformationO; 

adjustCompositSizeO; 

setSamplelnfomnationO; 

setUserlnformationO; 

} 

function getScreenlnformationO { 

scrn_height = Browser.GetScreenlnfoCheighO; 
scrnwidth = browser.GetScreenlnfoC'width"); 
scrn_ratio = Browser.GetScreenlnfoCratio"); 
scrn.scale = Browser.GetScreenlnfoCscale"); 
if (scrnjieight = 1080) { 
kx=1.00; 
ky=1.00; 



} else { 



if (scrn ratio = 1) { 
kx = 0.375; 
ky = 0.444; 



} else { 



kx = 0.375; 
ky = 0.333; 



1 



) 

function getSamplelnformationO { 
smpLwidth = 1024; 
srnplheight = 768; 
smpl.scale = xxxx; 
smpLguidex = 400; 
smpLguidey = 1 50; 
if ( scm height = 480) { 



B 
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FIG. 25 



sampLwidth = smplwidth kx; 
smpLheight = smpLheight ky; 
smpl ^uidex = smpLguidex kx 
smpLguidey = smpLguidey ky; 
smpLscale = smpLscale*ky; 



B 



function getUserlnformationO { 

usr.source = Browser.GetUserlnfo( face ); 
usr.scale = documentall.usr_source,style.scale* ky; 
var factor = smpl_scale/usr_scale; 

usr.height = document.all.usr_source.style.height*ky* factor; 
usrlwidth = documentall.usr.source.style.width* kx factor 
usr_guidex = documentall.usr_source.style.guidex* kx* factor 
usr ^guidey = documentall.usr_source.style.guidey* ky* factor; 

1 

function resizeSampleInformation( var f ) { 
smplwidth = smpl.width* f; 
smpl.height = smpLheight* f ; 
smpLguidex = smpLguidex *f; 
smpLguidey = smpLguidey *f ; 

} 

function resizeUserInformation( var f ) { 
usr.width = usr_width*f; 
usr.height = usrjieight*f ; 
usr .guidex = usr jjuidex*f; 
usr jjuidey = usr_guidey*f ; 

) 
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FIG. 26 



function ac(justCompositionSizeO { 

var compsite.width, remainx, factorx; 
var compsitejieight, remainy, factory; 
var resize; 

compsitej/vidth - 0; 

offsetx = smpLguidex - usr_guidex; 

if (offsetx<0) 

composite_width = composite jwdth - offsetx; 

} else { 

offsetx = 0; 

) 

remainx = offsetx + usr_width - smpl.width; 
if ( remainx < 0 ) { 

compsftwidth = comosite_width - remainx; 

factorx = smpLwidth/(compsite_width + smpLwidth); 

compsitejieight = 0; 

offsety = smpLguidey - usr_guidey; 

if ( offsety < 0 ) { 

composite Jieight = composite Jiight - offsety; 

} else { 

offsety = 0; 

} 

remainy = offsety + usrjieight - smpljieight; 
if ( remainy < 0 ) { 

compsit.height = comosrtejieight - remainy; 

factory = smpLheight/(compsrte Jieight smpl_height); 

if ( factorx < 1 ) 

resize = factorx; 
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FIG. 27 



resizeUserInformation( resize ); 
offsetx = (0 - offsetx)* resize; 
offsety = (0 - offsety)* resize; 



} 

function setSamplelnformationO { 

documentalhsampleDiv.style.width = smpl.width; 
documentall.sampleDiv.style.height = smpLheight; 
documentall.sampleDiv.style.top = smpltop + offsety; 
documerrt.all.sampleDiv.styte.left = smpljeft + offsetx; 

} 

function setllserlnformationO { 

documentall.UserDiv.style.width = usr_width; 
document.all.UserDiv.sty!eiieight = usrjieight; 
document.all.UserDiv.style.top = smpLtop + smpl_guidey - usr_guidey; 
documentall.UserDiv.style.left = smpljeft + smpLguidex - usr_guidex; 

} 

function setLivelnformationO ( 
if ( scroheight = 1080){ 
document.all.LiveDiv.style 

= style="position:absolute; top:20; left:1088; width:752 height*432 visibility:visible; 
} else { 
if( ratio =1)1 
documentalLLiveDiv.style 

= style="position:absolute; top:20; Ieft428; width:282 height192 visibi!ity:visible; 
} else { 
documentall.LiveDiv.style 

= style="position:absolute; top:20; Ieft428; width:282 height 144 visibilhyrvisible; 

} 

} 
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FIG. 28 



function setButtonlnformationO { 
if(scrn_hight= 1080)[ 

document.all.divJ>tn_up.style m 

= "positiomabsolute; top:472; Ieft1288; width:250 height200 visibihty:visible; 
documentall.divbtnjeftstyle m 

= "position:absolute; top:547; left 1088; width:200 height:250 visibility:visible; 
documentall.divJ>tn.down.style m 

= "position:absolute; top:672; left 1288; width:250 height200 visibilitytvisible; 
documentall.divjrtnj'ightstyle m 

= "positionrabsolute; top:547; left:1538; width:200 height:250 visibilrtyrvisible; 
documentaIl.div„btnback2.style m 

= "positiomabsolute; top:892; Ieft1281; width:264 height162 visibilityrvisible; 

} else { 

documentall.div>tn_up.styIe 

= "position:absolute; top220; Ieft520; width:100 height:80 visibilityrvisible; 
documentall.div.btnjeftstyle 

= "position:absolute; top:260; Ieft440; width:80 height 100 visibility:visible; 
documentalLdiv.btn_down.style 

= "positiomabsolute; top:320; Ieft520; width:100 height80 visibility:vistble; 
documentall.divjstnjightstyle 

= "position:absolute; top:260; Ieft620; width:80 height:100 visibility^isible; 
documentall.divJ>tnback2.style 

= "position:absolute; top:410; Ieft520; width:100 heightOO visibilityrvisibie; 

} 

} 

function move.upO { 

var top = UserDiv.style.top - 10; 
if (top < 60) { 

top = 60; 

} 

documentall.UserDiv.style.top = top; 

} 

function move.downO { 

var top = UserDiv.style.top + 10; 
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FIG. 29 



var limit = 60 + documentall.SampleDiv.style.height; 
if (top > limit)! 

top = limit; 

] 

documentall.UserDiv.style.top = top; 

} 

function moveJeftO { 

var left = UserDiv.style.left - 10; 
if (top < 40) { 

top = 40; 

} 

documentall.UserDiv.style.left = top; 

} 

function movejightO { 

var left = UserDiv.style.left + 10; 

var limit = 40 + documentall.SampleDiv.style.width; 

if (left > limit) { 

left = limit 

) 

documentall.UserDiv.style.left = left; 

} 



</script> 
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FIG. 30 



</head> 

<body onLoad="initializeO"> 

<div id="sampleDiv" style="posrtion:abso!ute; visibility:visible;'*> 

<object id="sampleImg",type="image/jpeg"></object> 

</div> 

<div id="UserDiv" sty le="position:ab solute; visibility:visiblef > 

<object id="usr.source' y ,type=''image/jpeg*'></object> 



<div id-"LiveDiv"> 

<object id="Uve' , ' f type="video/mpeg2" data="0(r></ofcuect> 



<div id="div_move_up~> 

<a onSelect="move_upO"> 

<object id="btn_up",type=~image/jpeg" data="btn_upjpg"></otyect> 
</a> 

</div> ^ 

<div id="div_move_left #r > 

<a onSelect=~move_left()"> 

<object id="btnjeft" l type="image/jpeg" data="btnJeftjpg"></object> 
</a> 

</div> 

<div id="div_btn_down"> 

<a onSelect="move_down(T> 

<object id="btn_down",type="image/jpeg" data="btn_downjpg"></object> 
</a> 

</div> ^ 

<div id^divJrtnjighO 

<a onSelect="move_right()' r > 

<object id^btnj'ight^type^image/jpeg" data=''btnjightjpg"><7object> 
</a> 

</div> 

<div id="div_btnback2"> 

<a href="startup.htm"> 

<object id='btnback2",type= // image/jpeg / ' data="btn_backjpg"></ object> 
</a> 

</div> 
</body> 



</div> 



</div> 
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FIG. 31 



( START OF INITIAL SCRIPT) 

. 1 

S1 001 ^ BROWSER PERMITTING VIRTUAL FITTING? j > 



S1002- 



S1003- 



S1004- 



S1005 



S1006- 



S1007- 



S1008- 



ACQUIRE RESOLUTION, ASPECT RATIO, AND 
. SCALE OF DISPLAY 



ACQUIRE IMAGE DATA OF COMMODITIES, AND 
MATCH DATA WITH CONDITIONS OF DISPLAY 



NO 



ACQUIRE USER IMAGE AND USER INFORMATION 
FOR VIRTUAL RTTING, AND MATCH THE INFORMATION 
WITH CONDITIONS OF COMMODITY IMAGE DATA 




SYNTHESIZED IMAGE WITHIN RANGE OF 
COMMODITY IMAGE? 



, f N0 



ADJUST SIZE OF COMMODITY IMAGE AND 
USER IMAGE, AND SYNTHESIZING POSITION 



SET DISPLAY POSITION AND SIZE OF 
COMMODITY IMAGE 



SET DISPLAY POSITION AND SIZE OF USER IMAGE 



( END OF INITIAL SCRIPT ) 



YES 
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FIG. 32 



S1101 



S1102- 



S1103- 
S1104- 

S1105- 



S1107- 



( START OF INITIAL SCRIPT ) 



< 


< 


SET SCALE OF USER IMAGE IN X DIRECTION TO BE 
FACTOR x WHEN USER IMAGE IN +/- X DIRECTION 
OVEREXTENDS COMMODITY IMAGE 






SET SCALE OF USER IMAGE IN Y DIRECTION TO BE 
FACTOR y WHEN USER IMAGE IN +/- Y DIRECTION 
OVEREXTENDS COMMODITY IMAGE 


' 




SCALE resize = factor x 







<^ SCALE IN Y DIRECTION < resize? \ 


> 


YES 

1 


SCALE resize = factor y 






< 

r 


<^ SCALE CONTRACTION OF 


I EXPANSION REQUIRED ? ^> 



NO 



NO 



YES 



UPDATE COMMODITY IMAGE DATA AND 
USER IMAGE DATA 



( END OF INITIAL SCRIPT ) 
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FIG. 33A 
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FIG. 34A 

731 732 


SOFT AND LIGHT SWEATER ^ 










i ' L "CIUKIN y 



733 
734 



FIG. 34B 



SOFT AND-UGHT SWEATER 




( RETURN ) 



FIG. 34C 



SOFT AND LIGHT SWEATER >4P== , 








[ RETURN ] 
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FIG. 35A 

731 732 



DAZZLING EARRINGS / 






[ RETURN )- 
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FIG. 35B 



DAZZLING EARRINGS 








i i — ■ y nciuruM j 



FIG. 35C 



DAZZLING EARRINGS 










f RETURN ] 
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